feat: version-aware Connection abstraction for server scenarios#318
Conversation
Copies schema/{version}/schema.ts from the modelcontextprotocol spec
repo into src/spec-types/{version}.ts so the conformance suite can type
against draft spec versions before any SDK ships them.
npm run sync-schema -- <ref> refreshes the copies and records the spec
commit in src/spec-types/SOURCE.
Connection encapsulates how the conformance suite talks to a server-under-test for a given spec version: - connectStateful: 2025-x lifecycle. Thin adapter over the SDK Client (initialize handshake, session id, SSE handled by the SDK). - connectStateless: 2026-x lifecycle (SEP-2575). Raw fetch with per-request _meta + MCP-Protocol-Version header. Decoupled from the SDK so the suite can test draft spec versions before the SDK implements them. connectFor(specVersion) picks the implementation. RunContext bundles serverUrl, specVersion and a bound connect() for the runner to hand to each scenario. Nothing uses this yet; wiring follows in the next commit.
ClientScenario.run(serverUrl) becomes run(ctx: RunContext). The runner builds the context from --spec-version and the server URL; scenarios destructure ctx.serverUrl and otherwise behave identically. No scenario uses ctx.connect() yet, so behaviour is unchanged: 214/214 tests pass, all-scenarios.test.ts still drives the everything-server fixture exactly as before. Test files use a testContext(url) helper to construct a RunContext. The authorization-server scenario list is retyped to ClientScenarioForAuthorizationServer since those scenarios test an OAuth server, not an MCP server, and keep run(serverUrl).
22 carry-forward and lifecycle scenarios now go through the Connection
abstraction instead of connectToServer + SDK Client:
- tools.ts (8): list, call x6, with-progress
- prompts.ts (5)
- resources.ts (7): list, read x3, subscribe x2, not-found
- utils.ts (3): completion, ping, set-level
- json-schema-2020-12.ts, caching.ts, http-standard-headers.ts
Result types come from spec-types/{introducedIn}. Same scenario code
now passes under both --spec-version 2025-11-25 (SDK-backed stateful)
and --spec-version draft (raw stateless).
ToolsCallSampling/Elicitation/WithLogging and elicitation-* keep
connectToServer (need setRequestHandler/setLoggingLevel SDK surface);
they are tagged removedIn: DRAFT in the next commit. stateless.ts and
input-required-result.ts keep their sendRpc helper; migrating those is
deferred to the DRAFT-scenario coherence pass.
…import connectStateful now catches the SDK's McpError and rethrows as JsonRpcError so scenarios always see the same error class regardless of which Connection impl ran. ResourcesNotFoundError uses instanceof JsonRpcError instead of duck-typing. types.ts uses a normal top-level import for RunContext instead of an inline import() type.
The 'valid Host accepted' check was sending an initialize body, which a 2026 server returns 404 for. Probe body is now picked from ctx.specVersion: initialize for the stateful lifecycle, server/discover with _meta for the stateless lifecycle. The Host/Origin rejection check is unchanged since rejection happens before body parsing.
…Server Adds an in-memory dispatch client connected to the same McpServer the stateful path uses. Stateless requests for tools/call, resources/*, prompts/get and completion/complete that fall through the MRTR-specific handlers are routed to it, so the fixture serves the carry-forward scenarios under --spec-version draft without duplicating ~500 lines of tool/resource/prompt registrations. tools/list now merges the McpServer's tool list with the MRTR-only stubs so json-schema-2020-12 finds its tool. draft suite against the fixture: 36/39 (was 13/39). Remaining 3 are fixture-side SEP gaps (no SSE forwarding for progress, no SEP-2549 ttlMs, no SEP-2243 Mcp-Method validation in the stateless path).
commit: |
| export function connectFor( | ||
| specVersion: SpecVersion | ||
| ): (serverUrl: string) => Promise<Connection> { | ||
| return STATEFUL_VERSIONS.has(specVersion) | ||
| ? connectStateful | ||
| : connectStateless; | ||
| } |
There was a problem hiding this comment.
For reviewers: this is the primary abstraction to allow testing scenarios over both old and new protocol version.
| drainNotifications: () => unknown[]; | ||
| close: () => Promise<void>; | ||
| }; | ||
| async function getStatelessDispatchClient(): Promise<DispatchClient> { |
There was a problem hiding this comment.
Shim for the fact that the TS SDK doesn't support stateless servers yet, and we don't want to duplicate every tool/resource/prompt registration in the stateless HTTP branch.
For each stateless request, we spin up an SDK Client + McpServer pair connected over InMemoryTransport. The in-memory client.connect() does the initialize handshake the SDK requires, then we forward the incoming method/params through it. The express handler writes whatever comes back to the HTTP response, so from the outside the fixture looks like a native stateless server.
Goes away once the SDK has stateless server support.
00b9f55 to
2b8b15c
Compare
| } | ||
|
|
||
| if (method === 'tools/list') { | ||
| const dispatch = await getStatelessDispatchClient(); |
There was a problem hiding this comment.
Note: the other changes in everything-server here sit inside the if (!session && (reqVersion || meta)) { ... } block which only applies in stateless mode.
A stateful request falls straight through to transport.handleRequest() unchanged.
I think it would be nice to refactor everything-server to have just 2 paths though, handleStateful and handleStateless, but didn't want to mix that in here.
| export interface RunContext { | ||
| serverUrl: string; | ||
| specVersion: SpecVersion; | ||
| /** | ||
| * Open a version-appropriate connection to the server-under-test. | ||
| * Scenarios that test the connection mechanics themselves (initialize, | ||
| * GET-SSE, DNS rebinding) bypass this and use raw fetch. | ||
| */ | ||
| connect(): Promise<Connection>; | ||
| } |
There was a problem hiding this comment.
instead of just a server URL, this is now the context for running a test in so the version can be specififed.
| return { | ||
| jsonrpc: '2.0', | ||
| id: 1, | ||
| method: 'server/discover', |
There was a problem hiding this comment.
small change to make the dns rebinding check compatible with stateless
| Vendored copies of `schema/{version}/schema.ts` from the | ||
| [modelcontextprotocol](https://github.com/modelcontextprotocol/modelcontextprotocol) | ||
| spec repository. | ||
|
|
||
| These are the canonical TypeScript types for each protocol version. The | ||
| conformance suite imports types from here rather than from | ||
| `@modelcontextprotocol/sdk` so that it can test draft spec versions before any | ||
| SDK has implemented them. | ||
|
|
||
| **Do not edit these files by hand.** To refresh: |
There was a problem hiding this comment.
This one we could argue about - I chose to import spec types directly here instead of relying on the typescript-sdk types to decouple the two.
Because the conformance tests really need to front-run SDKs, coupling to the typescript-sdk feels like the wrong approach here - it also matches conceptually more closely that SDKs are downstream of conformance which itself is downstram of the protocol.
Also given we now have different types we potentially need to handle (as some are removed / changed between 2025-11-25 and DRAFT) I'm thinking we probably can't get away with just having a single import anymore like we did?
Open to discussion on this one though.
…d tests Addresses self-review findings on the new connection module: - RequestOptions.handlers/.meta and ServerRequestHandler removed: zero callers (the scenarios that motivated them are deferred). They can be reintroduced when something actually uses them. - scenarios/server/client-helper.ts moved to connection/sdk-client.ts so the connection module no longer depends on the scenarios module. - connectStateless: handle CRLF SSE event separators; throw a useful error for non-JSON/non-SSE responses instead of a JSON parse error. - JSONRPCNotification consistently imported from spec-types/2025-11-25. - New connection.test.ts (8 tests) covering connectFor selection, _meta injection, error mapping, SSE LF/CRLF parsing, and the server-request-on-stream rejection.
- connectStateless: throw on non-2xx responses that lack a JSON-RPC error envelope (e.g. gateway/framework error JSON), matching the stateful path's behavior. Previously such a response would return undefined as the result. - json-schema-2020-12: rename negotiatedVersion to targetVersion and reword the skip message and details field. The value is ctx.specVersion (the run's --spec-version), not a negotiated version; the old wording was misleading. Drop dead 'unknown' fallback (specVersion is required) and the corresponding undefined test cases.
…est scaffolding Concurrent POSTs each answered with JSON or their own SSE stream are core transport behavior in the draft spec too, so removedIn was wrong for this scenario. Only the request scaffolding is version-specific: stateful runs keep the initialize/session-id setup, draft runs send _meta + the MCP-Protocol-Version header on each request instead. The two checks are unchanged.
|
Audited every Brought back:
|
| Scenario | Why removedIn is correct |
Where the behavior went |
|---|---|---|
server-initialize |
initialize is gone from the draft schema; lifecycle.mdx defines a stateless protocol, and protocol-level sessions are removed with it |
server-stateless covers server/discover + per-request _meta |
ping |
ping is gone from the draft schema; the ping utility page no longer exists in draft |
no replacement |
logging-set-level |
logging/setLevel is gone from the draft schema; logging.mdx specifies per-request _meta log level instead |
positive _meta.logLevel scenario still to write (gap table) |
resources-subscribe / resources-unsubscribe |
both RPCs gone from the draft schema; subscriptions.mdx: subscriptions/listen "replaces the former resources/subscribe RPC and the HTTP GET" |
server-stateless covers listen/acknowledge/list-changed; resource-updated path still to write (gap table) |
tools-call-with-logging |
depends on logging/setLevel, which is gone |
same _meta.logLevel gap as above |
tools-call-sampling |
asserts the server sends sampling/createMessage as a JSON-RPC request on the response stream; transports.mdx: the server "MUST NOT send independent JSON-RPC requests on this stream", server-to-client interactions are embedded per SEP-2322 |
input-required-result-basic-sampling |
tools-call-elicitation |
same MUST NOT, for elicitation/create |
input-required-result-basic-elicitation |
elicitation-sep1034-defaults |
same MUST NOT (delivery mechanism); the schema content (default on requested schemas) survives unchanged |
MRTR sibling asserting defaults on inputRequests[*].params.requestedSchema still to write (gap table) |
elicitation-sep1330-enums |
same, for enum constraints |
MRTR sibling still to write (gap table) |
server-sse-polling |
asserts SEP-1699 resumability (priming event, id:/retry: fields, Last-Event-ID reconnect); transports.mdx: "Resumable SSE streams via Last-Event-ID are not supported" |
no replacement |
The distinction I used throughout: a tag is only correct when the asserted behavior is gone from the draft, not just the test's setup. dns-rebinding-protection and now server-sse-multiple-streams are the two cases where the behavior survives and only the setup needed to become version-aware; both stay selected under draft.
MockServer encapsulates the lifecycle scaffold a client-conformance scenario presents to the client-under-test: - createServerStateful: 2025-x lifecycle. SDK Server + StreamableHTTPServerTransport (sessionless mode); the SDK handles the initialize handshake. - createServerStateless: 2026-x lifecycle (SEP-2575). Raw express app that validates _meta + MCP-Protocol-Version on every request, serves server/discover, routes other methods to the supplied handlers. createServerFor(specVersion) picks the implementation. ScenarioContext bundles specVersion and a bound createServer() for the runner to hand to each scenario. This is the client-conformance mirror of src/connection (PR #318). Nothing uses it yet; wiring follows in the next commit.
Conflicts resolved: - .prettierignore: keep both additions (spec-types/*.ts + tooling dirs) - src/scenarios/server/stateless.ts: keep both new imports - src/scenarios/server/resources.ts: keep PR318's Connection abstraction for SEP-2164; under --spec-version draft ctx.connect() resolves to connectStateless which is wire-equivalent to main's sendStatelessRequest - src/scenarios/server/http-standard-headers.ts: keep main's stateless discovery (removes obsolete initialize/session preamble that PR318 acknowledged as incoherent under draft); keep PR318's RunContext signature - src/scenarios/server/caching.ts: keep main's queryEndpoint refactor but drive it through PR318's conn.request() instead of sendStatelessRequest Semantic fix: - negative.test.ts: pass DRAFT_PROTOCOL_VERSION to testContext() for the SEP-2164 and SEP-2549 tests, since main rewrote those fixture servers to be stateless-only and the scenarios are draft-only anyway Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The two stateless request paths were ~90% duplicated: - scenarios/server/stateless-client.ts (main #319): buildStandardHeaders, withRequestMeta, readSseJsonRpcResponse, sendStatelessRequest -> raw {status, body}; sends Mcp-Method/Mcp-Name (SEP-2243); has timeout - connection/stateless.ts (this PR): connectStateless -> Connection.request -> throws JsonRpcError; missing Mcp-Method/Mcp-Name; no timeout connection/stateless.ts now exports both layers: - low-level sendStatelessRequest() (and the building blocks) moved verbatim from stateless-client.ts - connectStateless() rebuilt as a thin wrapper over sendStatelessRequest(): classifies SSE events into the notification sink, surfaces server->client requests on the response stream as a spec violation, throws JsonRpcError on error responses So connectStateless() now also picks up the SEP-2243 headers and the 10s timeout that the previous impl lacked. stateless-client.ts deleted; its 4 importers + test file repointed at '../../connection'. Test file moved to connection/stateless.test.ts. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ard dispatch The merge of main left two competing handler sets for resources/read, resources/list and resources/templates/list: the inline stateless handlers (test://stateless-static-text + a catch-all -32602) and the carry-forward dispatch to the McpServer-registered resources. The catch-all made the carry-forward unreachable, so test://static-text, test://static-binary and the templates errored under --spec-version draft. Now resources/list and resources/templates/list dispatch to the McpServer and merge with the inline entries (same pattern as tools/list and prompts/list), and resources/read keeps the inline static-text response but falls through to the carry-forward for unknown URIs (preserving SEP-2164 data.uri on errors). Also wrap the tools/list and prompts/list dispatch in try/finally so dispatch.close() always runs, matching tools/call. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ess requests sse-multiple-streams and dns-rebinding bypass sendStatelessRequest() because they need raw fetch/undici for low-level inspection, but their hand-built draft-path headers omitted Mcp-Method, so a strictly-conformant draft server would 400 them before the behaviour under test runs. Build the headers via buildStandardHeaders() instead, layering the scenario-specific overrides (Accept ordering, Host/Origin) on top. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
When --spec-version is omitted, the runner defaulted to LATEST_SPEC_VERSION, so draft-only scenarios (caching, resources-not-found, ...) silently ran over the stateful connection instead of the stateless draft transport they were written for; all-scenarios.test.ts had the same problem via testContext(). Infer the spec version from scenario.source.introducedIn (as negative.test.ts already does) when no explicit version is given. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…jects
connectStateless() destructured response.body.error unconditionally, so a
proxy returning {"error": "upstream timeout"} produced a JsonRpcError with
undefined code/message and lost the HTTP status. Require an object with a
numeric code before treating it as a JSON-RPC error; otherwise surface the
HTTP status and raw body in the thrown Error. Adds a unit test for the
non-conformant 502 case.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…eless Two consistency nits from automated review: - Collected notifications were missing the jsonrpc field (the SDK's Zod parsing strips it), violating the JSONRPCNotification type contract. - McpError messages kept the SDK's "MCP error <code>: " prefix, so JsonRpcError.message differed between the stateful and stateless impls. Also restore progressToken filtering in tools-call-with-progress, which was lost when the scenario migrated from the SDK onprogress callback to conn.notifications: the server must echo the request's token back, and notifications for other tokens are now ignored. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…orce; plumb spec version to the wire Running a draft-only scenario with an explicit dated --spec-version (e.g. server-stateless --spec-version 2025-11-25) used to silently pass: the flag only affected ctx.connect(), which hand-rolling scenarios never call, so they sent draft requests regardless and the run looked green while testing nothing the flag claimed. Three changes: - buildStandardHeaders / withRequestMeta / sendStatelessRequest / connectStateless accept an optional specVersion (default: draft), and hand-rolling scenarios pass ctx.specVersion through, so requests declare the version the run was invoked with. - The runner skips (exit 0) when an explicit --spec-version falls outside the scenario's introducedIn/removedIn window, with a message instead of a misleading green run. - A new --force flag overrides the skip for deliberate mismatch probing; combined with the plumbing, a forced server-stateless run at 2025-11-25 now genuinely sends that version (8/20 checks pass against the everything-server instead of a vacuous 26/26). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pre-merge verification — PR #318 @ e5ec4c9Build & test integrity
E2E — stateful path (2025-11-25, must match main)
E2E — stateless path (the new code in this PR)
Scenario inventory
Packaged CLI (what npm actually ships)
|
Mirrors the server runner's policy from #318: an explicitly requested --spec-version outside a scenario's applicability window skips it (exit 0) unless --force; when --spec-version is omitted the version is inferred from the scenario's applicability instead of defaulting to the latest dated version.
* feat: vendor spec schema types per-version
Copies schema/{version}/schema.ts from the modelcontextprotocol spec
repo into src/spec-types/{version}.ts so the conformance suite can type
against draft spec versions before any SDK ships them.
npm run sync-schema -- <ref> refreshes the copies and records the spec
commit in src/spec-types/SOURCE.
* feat: add Connection abstraction and RunContext
Connection encapsulates how the conformance suite talks to a
server-under-test for a given spec version:
- connectStateful: 2025-x lifecycle. Thin adapter over the SDK Client
(initialize handshake, session id, SSE handled by the SDK).
- connectStateless: 2026-x lifecycle (SEP-2575). Raw fetch with
per-request _meta + MCP-Protocol-Version header. Decoupled from the
SDK so the suite can test draft spec versions before the SDK
implements them.
connectFor(specVersion) picks the implementation. RunContext bundles
serverUrl, specVersion and a bound connect() for the runner to hand to
each scenario.
Nothing uses this yet; wiring follows in the next commit.
* refactor: thread RunContext through ClientScenario.run
ClientScenario.run(serverUrl) becomes run(ctx: RunContext). The runner
builds the context from --spec-version and the server URL; scenarios
destructure ctx.serverUrl and otherwise behave identically.
No scenario uses ctx.connect() yet, so behaviour is unchanged: 214/214
tests pass, all-scenarios.test.ts still drives the everything-server
fixture exactly as before.
Test files use a testContext(url) helper to construct a RunContext.
The authorization-server scenario list is retyped to
ClientScenarioForAuthorizationServer since those scenarios test an
OAuth server, not an MCP server, and keep run(serverUrl).
* refactor: migrate server scenarios to ctx.connect() + conn.request()
22 carry-forward and lifecycle scenarios now go through the Connection
abstraction instead of connectToServer + SDK Client:
- tools.ts (8): list, call x6, with-progress
- prompts.ts (5)
- resources.ts (7): list, read x3, subscribe x2, not-found
- utils.ts (3): completion, ping, set-level
- json-schema-2020-12.ts, caching.ts, http-standard-headers.ts
Result types come from spec-types/{introducedIn}. Same scenario code
now passes under both --spec-version 2025-11-25 (SDK-backed stateful)
and --spec-version draft (raw stateless).
ToolsCallSampling/Elicitation/WithLogging and elicitation-* keep
connectToServer (need setRequestHandler/setLoggingLevel SDK surface);
they are tagged removedIn: DRAFT in the next commit. stateless.ts and
input-required-result.ts keep their sendRpc helper; migrating those is
deferred to the DRAFT-scenario coherence pass.
* fix: normalize Connection error to JsonRpcError; clean up RunContext import
connectStateful now catches the SDK's McpError and rethrows as
JsonRpcError so scenarios always see the same error class regardless
of which Connection impl ran. ResourcesNotFoundError uses
instanceof JsonRpcError instead of duck-typing.
types.ts uses a normal top-level import for RunContext instead of an
inline import() type.
* fix(dns-rebinding): use version-appropriate probe body
The 'valid Host accepted' check was sending an initialize body, which
a 2026 server returns 404 for. Probe body is now picked from
ctx.specVersion: initialize for the stateful lifecycle, server/discover
with _meta for the stateless lifecycle. The Host/Origin rejection check
is unchanged since rejection happens before body parsing.
* feat(everything-server): route stateless carry-forward methods to McpServer
Adds an in-memory dispatch client connected to the same McpServer the
stateful path uses. Stateless requests for tools/call, resources/*,
prompts/get and completion/complete that fall through the MRTR-specific
handlers are routed to it, so the fixture serves the carry-forward
scenarios under --spec-version draft without duplicating ~500 lines of
tool/resource/prompt registrations.
tools/list now merges the McpServer's tool list with the MRTR-only
stubs so json-schema-2020-12 finds its tool.
draft suite against the fixture: 36/39 (was 13/39). Remaining 3 are
fixture-side SEP gaps (no SSE forwarding for progress, no SEP-2549
ttlMs, no SEP-2243 Mcp-Method validation in the stateless path).
* refactor(connection): drop unused RequestOptions; move sdk-client; add tests
Addresses self-review findings on the new connection module:
- RequestOptions.handlers/.meta and ServerRequestHandler removed: zero
callers (the scenarios that motivated them are deferred). They can be
reintroduced when something actually uses them.
- scenarios/server/client-helper.ts moved to connection/sdk-client.ts so
the connection module no longer depends on the scenarios module.
- connectStateless: handle CRLF SSE event separators; throw a useful
error for non-JSON/non-SSE responses instead of a JSON parse error.
- JSONRPCNotification consistently imported from spec-types/2025-11-25.
- New connection.test.ts (8 tests) covering connectFor selection,
_meta injection, error mapping, SSE LF/CRLF parsing, and the
server-request-on-stream rejection.
* fix: address bughunt findings (response.ok check; targetVersion naming)
- connectStateless: throw on non-2xx responses that lack a JSON-RPC
error envelope (e.g. gateway/framework error JSON), matching the
stateful path's behavior. Previously such a response would return
undefined as the result.
- json-schema-2020-12: rename negotiatedVersion to targetVersion and
reword the skip message and details field. The value is ctx.specVersion
(the run's --spec-version), not a negotiated version; the old wording
was misleading. Drop dead 'unknown' fallback (specVersion is required)
and the corresponding undefined test cases.
* fix(sse-multiple-streams): keep scenario in draft; version-aware request scaffolding
Concurrent POSTs each answered with JSON or their own SSE stream are
core transport behavior in the draft spec too, so removedIn was wrong
for this scenario. Only the request scaffolding is version-specific:
stateful runs keep the initialize/session-id setup, draft runs send
_meta + the MCP-Protocol-Version header on each request instead. The
two checks are unchanged.
* feat: add MockServer abstraction and ScenarioContext
MockServer encapsulates the lifecycle scaffold a client-conformance
scenario presents to the client-under-test:
- createServerStateful: 2025-x lifecycle. SDK Server +
StreamableHTTPServerTransport (sessionless mode); the SDK handles the
initialize handshake.
- createServerStateless: 2026-x lifecycle (SEP-2575). Raw express app
that validates _meta + MCP-Protocol-Version on every request, serves
server/discover, routes other methods to the supplied handlers.
createServerFor(specVersion) picks the implementation. ScenarioContext
bundles specVersion and a bound createServer() for the runner to hand
to each scenario.
This is the client-conformance mirror of src/connection (PR #318).
Nothing uses it yet; wiring follows in the next commit.
* refactor: thread ScenarioContext through Scenario.start()
Scenario.start() becomes start(ctx: ScenarioContext). The runner builds
the context from --spec-version (defaulting to LATEST_SPEC_VERSION) and
passes it through; scenarios receive it as _ctx and otherwise behave
identically.
No scenario uses ctx.createServer() yet, so behaviour is unchanged:
231/231 tests pass.
Test files use a testScenarioContext() helper. The runner already
threads MCP_CONFORMANCE_PROTOCOL_VERSION to the spawned client process,
so the fixture-side env wiring is unchanged.
* refactor: migrate tools_call to ctx.createServer; tag 2025-only client scenarios
ToolsCallScenario now goes through ctx.createServer() instead of an
inline express + SDK Server build. Same handlers, same checks; the
assertion now reads from srv.recorded so it works regardless of which
lifecycle scaffold the runner picked.
initialize, sse-retry, and elicitation-defaults are tagged
removedIn: DRAFT (initialize/GET-SSE/SSE-embedded-elicitation are gone
in the 2026 lifecycle; the MRTR sibling for elicitation-defaults is a
follow-up).
spec-version.test.ts: the 'draft is a superset of latest' invariant no
longer holds once removedIn: DRAFT exists; the test now asserts that
any scenario in latest-but-not-draft is explicitly removedIn.
* feat(auth): make createServer helper version-aware via ScenarioContext
The auth helper now takes ctx: ScenarioContext as its first argument
and branches on ctx.specVersion inside the /mcp route: the stateful
path (SDK Server + StreamableHTTPServerTransport) is unchanged; under
the draft version a raw stateless handler validates _meta + the
MCP-Protocol-Version header, serves server/discover, and routes the
same tools/list and tools/call responses.
The PRM endpoint, bearer-auth middleware, and request logger sit above
the branch and are version-independent.
All 25 call sites across the 12 auth scenario files pass ctx through;
ServerLifecycle and the express.Application return type are unchanged
so stop()/getChecks() are untouched.
Deviation from the MockServer wrapper approach: keeping the helper's
return type as express.Application avoids restructuring 25 call sites'
ServerLifecycle handling in this PR. Folding the auth seam onto
ctx.createServer() fully is a follow-up once the lifecycle ownership
moves into MockServer.
* feat(everything-client): pick stateless requester by MCP_CONFORMANCE_PROTOCOL_VERSION
Adds a statelessRequest(serverUrl, method, params) helper that POSTs
with _meta + MCP-Protocol-Version (the SEP-2575 lifecycle), shimming
around the SDK Client not yet supporting stateless mode. The
runRequestMetadataClient handler's meta constants are extracted to share
with the helper.
runBasicClient (initialize, tools_call, json-schema-ref-no-deref) now
branches on MCP_CONFORMANCE_PROTOCOL_VERSION: for the draft version it
uses statelessRequest to call tools/list then tools/call; for dated
versions it keeps the SDK Client path.
The runner already passes MCP_CONFORMANCE_PROTOCOL_VERSION to the
spawned client, so no runner change is needed.
* fix: address review findings on MockServer (dead opts, shared validator, capability derivation, recorded parity, specVersion threading)
- MockServerOptions removed (capabilities/configure had zero callers); opts
param dropped from createServerStateful/Stateless/For and ScenarioContext.
- validateStatelessRequest extracted from mock-server/stateless and exported;
both the stateless MockServer and auth/helpers/createServer.ts call it so
_meta/header/version validation cannot drift.
- isStatefulVersion exported from connection/select; mock-server/select uses
it instead of duplicating the version set.
- runner/client.ts: env MCP_CONFORMANCE_PROTOCOL_VERSION set unconditionally
to the resolved version; runInteractiveMode now takes specVersion and the
CLI passes it.
- createServerStateful: capabilities derived from handler method prefixes;
newServer() moved inside the try so a capability mismatch surfaces as
JSON-RPC -32603 instead of an HTML 500. Recording moved to the express
layer so unregistered methods are captured (parity with stateless).
- readFinalSseMessage return type now declares error.data.
- Tests added for the capability derivation and unregistered-method recording.
* fix(mock-server): record stateless requests before validation; document preamble exclusions
The stateless impl pushed to recorded[] only after validateStatelessRequest
passed, so requests rejected for a missing header or _meta never appeared,
diverging from the stateful impl which records at the HTTP layer before SDK
dispatch. Record before validation, excluding only the server/discover
lifecycle preamble (mirroring the stateful initialize/notifications-initialized
exclusion), and fix the MockServer.recorded docstring which wrongly claimed
stateless excludes nothing.
* refactor(mock-server): tri-state result for validateStatelessRequest (reject/handled/route)
A successful server/discover was previously modeled as {ok: false},
which read as a validation failure but actually meant "already
answered, do not route". Replace the boolean discriminant with an
explicit kind: 'reject' (validation failed), 'handled' (valid and
already answered), or 'route' (dispatch to handlers).
Both consumers (the stateless /mcp route and the auth helper's
handleStateless) still write status/body for reject and handled and
route only on 'route', so responses are unchanged. Add direct unit
tests covering each branch of the new union.
* feat(runner): single-source the version→lifecycle mapping; export MCP_CONFORMANCE_LIFECYCLE
The stateless mock and the example client both hardcoded the draft
protocol version, so dating the draft (or adding a second stateless
version) would silently break them: the mock would keep rejecting
everything but the old constant, and the client would fall back to the
stateful path.
- Derive STATELESS_SPEC_VERSIONS and a lifecycleFor() helper from
isStatefulVersion() in connection/select.ts, making it the single
source of truth for the version→lifecycle mapping.
- Thread the resolved spec version into the stateless mock:
createServerFor() binds it into createServerStateless(), and
validateStatelessRequest() now takes the supported-versions list as a
parameter. A mock bound to a version accepts exactly that version and
echoes it in -32004 rejections and server/discover; unbound mocks
accept every known stateless version.
- Export MCP_CONFORMANCE_LIFECYCLE (stateful|stateless) to client
processes alongside MCP_CONFORMANCE_PROTOCOL_VERSION, in both the
runner and the in-process test helper, so clients in any language can
pick the lifecycle without keeping their own version map.
- everything-client: import DRAFT_PROTOCOL_VERSION from src/types
instead of duplicating the string, branch on MCP_CONFORMANCE_LIFECYCLE
(falling back to the version comparison for older runners), and send
the runner-resolved version on stateless requests.
- Document MCP_CONFORMANCE_LIFECYCLE in the README and ignore the local
.claude/ workspace dir in .prettierignore so prettier --check . is not
tripped by untracked worktree checkouts.
* fix(everything-client): list-only flow for json-schema-ref-no-deref
runBasicClient now follows tools/list with a tools/call on the first
listed tool, but the json-schema-ref-no-deref scenario mock only serves
tools/list, so the call got -32601 and the client exited non-zero.
Register the scenario with a dedicated handler that stops after listing
tools, keeping the existing lifecycle branch (raw stateless request vs
SDK client). Listing is the whole flow for SEP-2106: the check is that
the client never fetches the network $ref canary while processing the
advertised schema.
* fix(tools_call): make getChecks() idempotent
getChecks() pushed into an instance array on every call, so calling it
more than once accumulated duplicate check entries. The runner is
allowed to call getChecks() repeatedly, and other scenarios already
build their result fresh per call for this reason.
Return a freshly built array instead and drop the now-unused instance
field. Add a unit test covering the not-called FAILURE case and
asserting repeated getChecks() calls do not grow the result.
* fix: ensure dispatch and transport cleanup runs when handlers throw
The stateless tools/list and prompts/list handlers in the everything
server awaited dispatch.close() after the in-memory request, so a throw
from dispatch.client.request() leaked the Client+Server+Transport pair.
Wrap both in try/finally, matching the tools/call and catch-all
handlers in the same file.
In the stateful mock server, the res 'close' listener that tears down
the per-request SDK transport and server was registered after the
handleRequest await; a throw there skipped registration and leaked the
pair. Register the listener before handing the request to the
transport.
* fix(everything-client): send Accept header on stateless requests
The draft transport spec requires POST requests to accept both
application/json and text/event-stream. Servers built on the SDK's
StreamableHTTPServerTransport already enforce this and reject
requests without it. Does not yet make json-schema-ref-no-deref pass
under --spec-version draft: the SDK transport also rejects the draft
protocol version itself, which is the known fixture gap until the
2026-native scenarios migrate onto MockServer.
* chore: exclude .claude/ from vitest globs
Local agent worktree checkouts under .claude/worktrees/ carry their
own copies of the test suite, which the bare **/*.test.ts include was
adding to every run (and to the pre-push hook). Mirrors the existing
.prettierignore entry.
* fix(request-metadata): use spec error code -32004 for unsupported protocol version
The draft spec defines -32004 (UnsupportedProtocolVersionError) with
data { supported, requested } and HTTP 400 for protocol version
rejections; -32001 (HeaderMismatch) is reserved for HTTP header
validation failures.
- request-metadata scenario: return -32004 with spec-shaped data for
the simulated version-negotiation rejection (was -32001)
- everything-client: trigger negotiation on -32004 (was -32001)
- stateless mock server: include the requested version in the -32004
error data alongside the supported list
- update tests to match and assert the new requested field
* fix(auth): register transport cleanup before handleRequest in createServer
The res 'close' listener that tears down the transport/server pair was
registered after awaiting handleRequest, so a throw mid-request skipped
cleanup registration entirely. Register it first, matching the stateful
mock server.
* fix(everything-server): return JSON-RPC errors from stateless list dispatch on failure
The stateless tools/list and prompts/list dispatch handlers had
try/finally but no catch, so a dispatch failure produced an unhandled
rejection instead of a response. Add the same catch-and-respond block
the carry-forward dispatch handler already uses.
* refactor(runner): drop MCP_CONFORMANCE_LIFECYCLE; derive lifecycle from protocol version
The protocol version fully determines the lifecycle, and deriving it is
part of being a conformant client. The in-repo example client imports
the stateless version set from src/, so there is no drift risk, and a
single env var cannot disagree with itself. README documents the
version→lifecycle mapping for clients in other languages.
* fix(everything-server): return JSON-RPC errors from stateless resource list dispatch on failure
resources/list and resources/templates/list arrived from main with
try/finally but no catch, unlike the sibling tools/list and
prompts/list handlers. A throwing dispatch would escape to Express's
HTML 500 instead of a JSON-RPC error. Mirrors the existing pattern.
* fix(everything-server): include requested field in -32004 error data
The spec's UnsupportedProtocolVersionError requires data to carry both
supported and requested. The mock-server and request-metadata -32004
producers already include it; this was the remaining producer.
* feat(runner): client-side spec-version inference, skip, and --force
Mirrors the server runner's policy from #318: an explicitly requested
--spec-version outside a scenario's applicability window skips it
(exit 0) unless --force; when --spec-version is omitted the version is
inferred from the scenario's applicability instead of defaulting to
the latest dated version.
* fix(auth): gate SEP-837 application_type check to spec versions that include it
The sep-837-application-type-present check fired on every Dynamic Client
Registration regardless of the targeted spec version, but SEP-837 only
exists in the draft spec. Clients conforming to 2025-11-25 or earlier were
failed on a requirement that is not part of the version under test.
Thread the ScenarioContext into createAuthServer (mirroring createServer)
and only emit the check when the resolved spec version includes SEP-837,
i.e. the stateless draft lifecycle. At dated versions the check is not
emitted at all. Audited the other auth helper checks (PKCE, metadata
discovery, token verification); all of them enforce requirements that are
already in the dated specs, so SEP-837 was the only draft-only check.
Unit tests cover both sides: the check fails an omitting client at the
draft version and is absent at 2025-11-25.
* feat(sdk-runner): --spec-version passthrough with per-SDK default
The sdk command forwarded --scenario/--suite/--timeout to the underlying
client/server invocation but had no way to set the spec version, so SDK
runs always used the suite default. Add a --spec-version flag that is
passed through to the inner run.
Also add an optional specVersion field to the per-SDK config, used as the
default when the flag isn't given; an explicit --spec-version overrides
it. Set typescript-sdk-v1 to 2025-11-25: the v1 line targets the latest
dated spec, which keeps draft-only scenarios out of its runs by default.
---------
Co-authored-by: Paul Carleton <paulc@anthropic.com>

Hoists the connection preamble out of individual server scenarios and into the runner, so the same scenario code runs under both the 2025 stateful lifecycle and the 2026 stateless lifecycle (SEP-2575).
Motivation and Context
Carry-forward behaviors (
tools/list,resources/read,prompts/get, etc.) are spec-invariant, but every server scenario hard-codedconnectToServer()→ SDKClient.connect()→initialize. That preamble is 2025-specific; in the 2026 draft it's replaced by per-request_meta+MCP-Protocol-Versionheader. So a scenario tagged{introducedIn: '2025-06-18'}with noremovedInwas selected under--spec-version draftbut couldn't actually exercise a pure-2026 server — it would fail at connect, or pass only because the fixture is dual-stack.This PR makes the connect preamble a function of
--spec-version, not something each scenario owns.What changes
src/spec-types/{version}.ts— vendored verbatim frommodelcontextprotocol/schema/{version}/schema.tsso the suite can type against draft spec versions before any SDK ships them.npm run sync-schema -- <ref>refreshes;SOURCErecords the pin.src/connection/—Connectioninterface (request<R>(method, params, opts?),notifications,close) with two impls:connectStateful— thin adapter over the SDKClient(don't reimplement the 2025 handshake/session/SSE)connectStateless— raw fetch with_metainjection +MCP-Protocol-Versionheader, decoupled from the SDKconnectFor(specVersion)picks the impl. Both throwJsonRpcErroron JSON-RPC error responses.ClientScenario.run(serverUrl)→run(ctx: RunContext)whereRunContext = {serverUrl, specVersion, connect()}. Runner builds it from--spec-version.connectToServer()+ SDK convenience methods toctx.connect()+conn.request<ResultType>('method', params). Result types come fromspec-types/{introducedIn}.removedIn: DRAFT— they test methods or mechanics removed in the 2026 draft.dns-rebinding-protectionnow picks its probe body fromctx.specVersion(initialize vs server/discover) so the "valid Host accepted" check works under both.everything-serverstateless path now dispatches carry-forward methods (tools/call,resources/*,prompts/get,completion/complete) to the sameMcpServerinstance the stateful path uses, via an in-memory client.tools/callis served as SSE so progress notifications reach the conformance client.How Has This Been Tested?
214/214 unit tests, typecheck, lint, build clean.
Full
--suite allagainsteverything-server:--spec-version2025-11-25draftThe one ✗ under draft is
http-header-validation, which is already inpendingClientScenariosListon main ("Pending until everything-server fully implements SEP-2243 header validation") and is unrelated to this change — the scenario itself sends a rawinitializeto obtain a session, which doesn't work under the stateless lifecycle. That's part of the broader DRAFT-scenario coherence pass.Per-scenario matrix (51 scenarios)
Applicable under both — 21
2025-only (
removedIn: DRAFT) — 11draft-only (
introducedIn: DRAFT) — 192026 coverage gaps created by
removedIn: DRAFTtaggingSome 2025-only scenarios test behavior that still exists in 2026 via a different mechanism. The
removedIntag is correct (the wire mechanic changed), but a 2026 sibling is needed to cover the same spec requirement:removedInscenarioserver-initializeserver/discover+_metaserver-statelesspinglogging-set-levellogging/setLevelaccepted_meta.logLeveltools-call-with-loggingnotifications/messageafter setLevel_meta.logLevelpresentresources-subscribe/-unsubscriberesources/subscribeacceptedsubscriptions/listentools-call-samplingsampling/createMessagevia SSEinputRequestsinput-required-result-basic-samplingtools-call-elicitationelicitation/createvia SSEinputRequestsinput-required-result-basic-elicitationelicitation-sep1034-defaultsdefault(SEP-1034)elicitation-sep1330-enumsenum(SEP-1330)server-sse-pollingLast-Event-ID(priming event,id:,retry:)Scenarios to write for full 2026 parity
tools-call-with-logging-metatools/callwith_meta['io.modelcontextprotocol/logLevel']: 'debug'set → server emitsnotifications/messageon the response stream at that levelserver-statelesscovers the negative (sep-2575-server-no-log-without-loglevel) but not the positive;sep-2575.yamlhas nocheck:row for "emits at requested level"sep-2575-server-emits-log-at-requested-level(new row insep-2575.yaml)subscriptions-resources-updatedsubscriptions/listenwithresourcesUpdated: {uris: [...]}→ server sendsnotifications/resources/updatedon the stream when the resource changesserver-statelesscoverssep-2575-server-tags-subscription-idandsep-2575-server-sends-{tools,prompts}-list-changed-on-subscription, but not the resource-content-update pathsep-2575-server-sends-resources-updated-on-subscription(new row insep-2575.yaml)mrtr-elicitation-sep1034-defaultstools/call→InputRequiredResult.inputRequests[k].params.requestedSchemacarriesdefaultvalues per SEP-1034input-required-result-basic-elicitationchecks the roundtrip but not schema-shape detailssep-1034-defaults-via-mrtr(nosep-1034.yamlexists yet)mrtr-elicitation-sep1330-enumsenumconstraints per SEP-1330sep-1330-enums-via-mrtr(nosep-1330.yamlexists yet)Adding the
check:rows to the SEP YAMLs would make these show asuntestedintraceability.json(the existing TODO mechanism); not done in this PR.Breaking Changes
ClientScenario.run(serverUrl: string)→run(ctx: RunContext). All in-tree scenarios are updated; out-of-tree scenarios (none known) would need a one-line shim.Types of changes
Checklist
Additional context
Deferred (out of scope, will conflict with the DRAFT-scenario coherence pass):
stateless.ts/input-required-result.tsoff theirsendRpchelper ontoConnection— those scenarios assert onerror.code/ HTTP status for nearly every call; would need anexpectErrorhelper orhttpStatusonJsonRpcError.http-header-validationcoherence under draft (it sends rawinitializeto get a session).src/scenarios/client/*) — symmetricctx.createServer()abstraction.Type import rule: a scenario imports result types from
spec-types/{its source.introducedIn}. That's the contract it asserts; if a later spec adds optional fields, the carry-forward scenario doesn't see them (a separate scenario covers the addition).